The VideoToolbox allows you to produce accurately controlled visual stimuli. This document deals with time; how to make sure each video frame shows what you want.
MOVIES. It is easy to show movies on a Mac. Create all your pixmaps in memory ahead of time, and then call either CopyBits() or CopyBitsQuickly() to copy one image after another to the screen. For any serious application you'll want a real-time movie, one image per video frame, so you'll want to wait for a new frame before copying each image, using one of the synchronization techniques described below. Look at the demos Sandstorm and NoiseVBL. Of course, how big a movie you can show in real time will depend on how fast your processor is. The demo TimeVideo will do this timing for you, telling you what fraction of the screen you can fill with a real-time movie. If you can't do a full-screen movie and you need to show multiple patches, consider showing several small movies, updating only the dynamic parts of the screen. (We assume that you are not limited by memory, and therefore do not discuss Apple's QuickTime, whose principal purpose, beyond making movies portable, is to minimize storage requirements by image compression.)
PAGE SWITCHING. Many video cards have multiple video pages; check your TimeVideo report. Normally the Macintosh only uses page 0, but the pages can be switched by calling GDSetPageShown() and GDSetPageDrawn(). This could be used to show very short full-screen movies, e.g. alternating two images, but I haven't had any occasion to try it.
CLUT (Color Lookup Table) ANIMATION. Most video cards have a hardware color lookup table (clut) that dynamically transforms each pixel to a color (an RGB triplet) that is sent to the digital to analog converters. Temporal modulation (e.g. flicker or fading on and off) of visual stimuli can be achieved very conveniently by loading a new clut on each frame. Look at the demo FlickeringGrating. However, not all video card drivers are fast enough to achieve this. (Usually this is not a hardware limitation; it's crumby driver software.) Run TimeVideo to find out for sure; it determines how many frames it takes either GDSetEntries or GDSetEntriesQuickly to load the clut. The clut is also used for gamma correction. Apple's provision for gamma correction is somewhat crude; for the utmost luminance accuracy you may wish to use the Luminance.c routines. See section B, below.
FANCIER TRICKS. The video card has only one clut, but it is possible to simultaneously present multiple independently modulated patterns, by reserving separate sections of the clut for each temporal modulation, though you'll suffer a comensurate loss of intensity resolution. Clut animation is not restricted to dynamic modulation of contrast. If the pixel values represent phase, and the clut is loaded with a sin function, then rotating the clut entries--i.e. setting the i-th entry to what was formerly the value of the i-1 entry--will shift the phase of the pattern. A spatially vignetted drifting grating may be synthesized by using alternate pixels (or frames) to represent vignetted sin and cos components of the pattern, devoting half the clut to each (or loading alternate cluts on alternate frames). Appropriate adjustment of the relative contrasts of the sin and cos components will produce any desired phase of the sum. However, most of these tricks are superfluous; it’s usually easier and better to just show a movie. Many current processors are fast enough to show 8-bit full-screen movies.
A. HOW TO SYNCHRONIZE A MACINTOSH PROGRAM TO A VIDEO CARD.
The hardest part in doing vision experiments is synchronizing the computer program to the video card. Nearly all video cards are masters, and run freely, expecting the video monitor to be a slave. Any experiment that cares about timing will normally have to synchronize itself to the video card, by waiting for each frame to end, as will be discussed below.
You typically can't access the video card hardware directly, because it's undocumented. Instead you are allowed to send requests to the video driver associated with your video card. All video cards that plug into the Mac come with video drivers that conform sufficiently well to the Apple guidelines that they are at least minimally compatible with the VideoToolbox. However, some drivers have outright bugs, and the timing of many drivers, for which Apple does not publish guidelines, sometimes makes it surprisingly difficult to do simple things. A current list of these bugs and "features" appears in section E, at the end of this document.
Every video card has a video driver in its ROM, but often the manufacturer supplies a newer driver on floppy that supercedes the one in ROM, which may be buggy. (Built-in video devices, e.g. in the Mac IIci and Quadra computers, load the video driver from the computer's ROM.) Apple distributes video driver updates as resources in the System file, so updating your System may change the video driver of your Apple video card. IdentifyVideo(device) returns a string with the name and version number of the driver that is actually in use. Try TimeVideo and TestGDVideo.
There are three different ways to synchronize a program to a video card, which we'll first describe quickly, and then more fully. (1) Apple recommends using the vertical blanking interrupts, which supposedly occur once per frame, at the beginning of the vertical blanking. The VideoToolbox VBLInstall.c routines make this very easy to do. A disadvantage of this approach is that you can't then raise the processor priority to block all other interrupts. (2) Most (not all) video drivers, when asked to load the clut, wait until the vertical blanking interval before beginning to load the clut. (They wait in order to avoid creating visible hash on the screen while the clut is changing.) This has the side effect of synchronizing your program to the display, since the driver doesn't return control until the VBL interval occurs. Either of these synchronization methods (1 or 2) may fail, depending on which video card you have, what you've set the pixel size to (1 to 32 bits), and whether you've raised the processor priority. The demo program TimeVideo tests both methods of synchronization on all your video cards at all pixel sizes and saves the results in a text file. (3) I think that all video cards provide a read-only vertical-blanking bit, but unfortunately few manufacturers will tell you its address, so you have to find it yourself (typically by disassembling the video driver), which is tedious.
1. VBL Interrupts. Most video cards emit one VBL interrupt during each video frame. Apple's new video cards ("Macintosh Display Card": 8•24) and the built-in video in the Quadra 700 and 950 generates several interrupts per frame. (Kyle Cave discovered that there are no extra interrupts if the cache is disabled on the Quadra 700.) This behavior is contrary to Apple's documentation, but VBLInstall.c works around the bug, using a timer to discard the excess interrupts (as suggested by Raynald Comtois). Some video drivers take multiple frames to load the clut, and block interrupts while doing so, wreaking havoc with any attempt to count frames. VBL interrupts are blocked while the processor priority is raised. Try the demos TimeVideo and NoiseVBL.
The best current theory on the source of the extra interrupts is that the card is asserting the interrupt line for a fixed duration. As long as you are in the interrupt handling routine (or one of higher priority), other interrupts of the same level are disabled. If you stay in the interrupt routine for long enough, the interrupt line has time to be released and you don't get a second interrupt, otherwise you do. (Suggested by Raynald Comtois.)
2. LOADING the clut. It seems that all video drivers automatically synchronize clut-loading to the video frame. Apple's guidelines for video drivers (in the book Designing Cards and Drivers) state that the video driver may save the clut-load (i.e. set-entries) request and defer the actual loading until the next VBL interrupt. Such video drivers return almost immediately after a set-entries request. Thus, although the actual clut load will be synchronized to the display, your program will be asynchronous because the GDSetEntries call will return immediately. However, Apple's guidelines state that if the processor's interrupt priority has been raised, suspending the video card's VBL interrupts, then the video driver should always load the clut before returning, which would synchronize your program to the display. You can raise and lower the processor priority by calling the VideoToolbox routine SetPriority.c. The priority is normally zero. TimeVideo measures timing at both normal and high priority.
Be aware that a set-entries request, i.e. calling GDSetEntries(), does not necessarily wait for the beginning of the next vertical blanking interval. It might merely wait until blanking is true. As a result two successive GDSetEntries calls might occur during the same blanking interval. To get exactly one call per frame you may need to delay for a suitable interval (perhaps 1 ms) before calling it again.
Most video drivers that I've tested seem to be synchronous and reliably take exactly one frame to load the clut. However, some video drivers take longer, e.g. any loading of the clut on Apple's new video cards ("Macintosh Display Card": 8•24) takes 30 ms--two frames--which is unacceptably slow for lookup table animation. The TrueVision NuVista seems to be asynchronous. Presumably the on-board processor accepts the clut information at any time and actually updates the clut during the next vertical blanking interval. However, this means that asking the video driver to load the clut doesn't have the useful side effect of synchronizing the Mac program to the display. The NuVista takes about 0.3 frames to reload the whole clut when in 8, 16, or 32-bit mode, but takes several frames when in 1, 2, or 4-bit mode (presumably because the fractional byte addressing is slow). I haven't checked, but presumably the NuVista driver would become synchronous, as specified by Apple, if the processor priority were raised.
SetEntriesQuickly.c written primarily by Raynald Comtois, Peter Lennie, and Bill Haake provides fast clut loading for many popular video devices. Try TimeVideo.
If 8-bit pixels are enough, consider buying Apple's old "Toby" or "TFB" video cards, since they work fine, and cost only $90 each from Shreve Systems (800)-227-3971.
3. BLANKING BIT. Apparently all video cards have a blanking bit (telling whether the video signal is in the blank period between frames), but, alas, the manufacturers never tell you where it is. You could figure out which bit by disassembling the driver, as I did for the Apple Toby and TFB video cards (no longer sold by Apple; see TFBVideo.c). Perhaps one could write a program that would find the VBL bit automatically, by looking for any bit that changes at the right frequency.
WaitForBlanking() in SetEntriesQuickly.c tests the blanking bit, but presently supports only the (obsolete) Toby and TFB video cards. Hopefully other people will enhance this routine to support more devices.
CONCLUSIONS. Don't take synchronization or lookup table animation for granted. Run TimeVideo to check out your video cards.
B. CONTROLLING THE COLOR LOOKUP TABLE (CLUT)
QuickDraw is one of the great virtues of the Macintosh. However, it makes several assumptions about what you want to do that are inappropriate for vision experiments. In particular, it assumes that you want all your monitors to act as one consistent desktop (with consistent color tables). This is a problem if you want to load completely independent lookup tables and images onto two monitors that, for example, you may want to superimpose optically. QuickDraw enforces the consistency throught the Palette Manager, but calls that are nominally to the Color Manager (e.g. Apple's SetEntries) may be intercepted by the Palette Manager, resulting in undesired effects on other screens. My solution is to bypass QuickDraw and to load the lookup tables more or less directly, without telling QuickDraw.
GDSetEntries() and SetEntriesQuickly() work outside of QuickDraw. GDSetEntries uses the video driver; SetEntriesQuickly goes directly to the hardware. They load the clut of the video card without changing the color spec table of the graphics device; QuickDraw will continue to assume that the graphics device's color spec table is a true copy of the clut. This is the behavior that I usually want. However, it may cause problems if you use CopyBits since CopyBits will translate the color of each pixel it copies, using the inverse color table of the current device, which is based on the color spec table, NOT the clut. There are several solutions to this. You can copy your color spec table into the graphics device's color spec table, and set ctSeed to alert QuickDraw that it's been changed. Or, instead of using QuickDraw's CopyBits and SetCPixel, you could use the VideoToolbox's CopyBitsQuickly and SetPixelsQuickly(), which copy pixel values directly, ignoring all color tables and inverse color tables, .
C. DISASSEMBLING A VIDEO DRIVER
Assuming you can read 680x0 assembly code, use the VideoToolbox utility GrabVideoDrivers to put the driver into a file, then use ResEdit with the ResEdit CODE editor to examine it, comparing it with the example in the appendix of Apple's Designing Cards and Drivers book. The ResEdit CODE editor is a public domain file distributed by:
The Mac IIci built-in video driver (.Display_Video_Apple_RBV1 driver, version 0) has a bug that causes it to crash if you try to do a getEntries Status request. Here are two ways to fix the bug.
1. AUTOMATIC TEMPORARY PATCH. The bug only affects GDGetEntries, so the first time GDGetEntries is invoked it automatically calls PatchMacIIciVideoDriver() in GDVideo.c to find and patch the copy of the buggy driver residing in memory, preventing any trouble. (If the driver is in ROM, then the driver is copied to RAM, and patched there.) Only two instructions are modified, to save & restore more registers. This fix persists only until the next reboot. The driver's version number is changed from 0 to 100, so that programs can distinguish it from the buggy version 0.
2. PERMANENT UPGRADE. The Mac IIsi has version 1 of the same driver, without the bug. In principle all you have to do is copy the new driver from the IIsi to your IIci, but this is nontrivial because these drivers are in ROM. It is fairly simple, as described below, to place a copy of the version 1 (i.e. fixed) driver into the System file of the Mac IIci, but, as explained in Inside Mac V-424, this will only displace the older version 0 (buggy) driver if it's not driving the boot monitor (i.e. if the "Welcome to Macintosh" message appears on some other monitor), because the boot monitor's driver is loaded at boot time before the System file is available. If you have multiple monitors, you can use the Monitors Control Panel to change the boot monitor: hold the option key down and drag the smiling Mac to any monitor except the one driven by the built-in video. So, if you do have multiple monitors, here's how to copy the driver. First put the VideoToolbox utility "GetVideoDrivers" on a floppy and run it on a Mac IIsi. That will copy the driver into a ResEdit file "-Display_Video_Apple_RBV1" on your floppy. Then put the floppy in your Mac IIci. Make a copy of your System file. Open the "-Display_Video_Apple_RBV1" file in ResEdit. Hit Command-I to edit the Resource Info. Set the "System Heap" and "Purgeable" flags, change the ID from 1 to 122, and close the Resource Info window. Now copy the resource and paste it into the copy of your System file. Quit, saving changes. Now rename the active System file to something else, like "old System", and rename the edited System file to "System". Reboot. You can now throw the old System into the trash. The new driver will automatically be favored over the one in ROM because it has a higher version number (1 instead of 0). All my test programs indicate that the new driver works fine in the Mac IIci with System 7, and I would expect it to work fine with System 6 as well. (Thanks to Mike.Alexander@um.cc.umich.edu for figuring out why it didn't work for the boot monitor.)
E. KNOWN BUGS IN VIDEO DRIVERS
1. The following drivers crash if one attempts to make a get-entries request. GDGetEntries tests for these driver names and version numbers in order to avoid these bugs. Try the demo TestGDVideo.
•Mac IIci “Macintosh II Built-In Video” (.Display_Video_Apple_RBV1)
(Reported to Apple MacDTS 3/7/90 and Apple.Bugs 12/30/92)
•Relax 19" Model 200 (.Color_Video_Display version 9288). (Reported to Relax Tech. 1/19/93)
2A. The following drivers take too long (more than 1 frame) to load the clut (i.e. set-entries request: GDSetEntries). Try TimeVideo.
•Apple 4•8 and 8•24 “Macintosh Display Card” (.Display_Video_Apple_MDC) and
“Macintosh Display Card 8•24 GC” (.Display_Video_Apple_MDCGC) take two frames to load the clut in all modes. (Reported to Apple.Bugs 12/30/92)
•Radius PrecisionColor 8 takes 8 frames to load the whole clut in 8-bit mode; the time is proportional to the number of clut entries being loaded. (Ok in 1-, 2-, and 4-bit modes.)
•“RasterOps ColorBoard 264” (.RasterOps 1.2 264 Video Driver version 9327) takes 2 frames to load the clut in 8- and 32-bit modes. (Ok in 1-, 2-, and 4-bit modes.)
•RasterOps “PaintBoard Li” 1.0 24XLi Video Driver takes four frames to load the clut in 8-,16-, and 32-bit modes. (Ok in 1-,2-, and 4-bit modes.) (Reported to RasterOps 12/30/92).
•TrueVision NuVista takes too long in 1-, 2-, and 4-bit modes. (Ok in 8-,16-, and 32-bit modes.)
2B. (This is closely related to 2A.) The following video drivers suppress one or more frame's worth of VBL interrupts each time they load the clut, i.e. perform a set-entries request: GDSetEntries. Try TimeVideo.
•Apple 4•8 and 8•24 “Macintosh Display Card” (.Display_Video_Apple_MDC) and
“Macintosh Display Card 8•24 GC” (.Display_Video_Apple_MDCGC) suppress one frame's worth. (Reported to Apple.Bugs 12/30/92)
•“RasterOps ColorBoard 264” (.RasterOps 1.2 264 Video Driver version 9327) suppresses one frame's worth in 8- and 32-bit modes.
•RasterOps “PaintBoard Li” 1.0 24XLi Video Driver suppresses three frames' worth of interrupts in 8-,16-, and 32-bit modes. (Reported to RasterOps 12/30/92).
3. The following cards and drivers all issue multiple VBL interrupts per frame, whereas they should emit only one per frame. Try TimeVideo.
•The 4•8 and 8•24 “Macintosh Display Card” (.Display_Video_Apple_MDC) emit, on average, about 2.5 VBL interrupts per video frame. (Reported to Apple.Bugs 12/30/92)
•Apple Quadra 700 "Macintosh E Built-In Video" (.Display_Video_Apple_DAFB) emits 2 VBL interrupts per frame if the processor cache is enabled, but emits only 1, as it's supposed to, if the processor cache is disabled. (Reported by Kyle Cave, cavekr@ctrvax.vanderbilt.edu)
•Apple Quadra 900 “Macintosh C Built-In Video” (.Display_Video_Apple_DAFB) occasionally emits more than 1 VBL interrupt per frame.
•Apple Quadra 950 “Macintosh G Built-In Video” (.Display_Video_Apple_DAFB version 2) emits 3 VBL interrupts per frame. (Reported to Apple.Bugs 12/30/92)
4. GDGetEntries !=GDSetEntries. The following drivers return garbage from a get-entries Status call following a set-entries Control call. Try TimeVideo.
•TrueVision NuVista GDDirectSetEntries (i.e. 16- and 32-bit pixel modes) seems to assume a zero “start”. GDSetEntries (i.e. 1-, 2-, 4-, and 8-bit pixel modes) works fine.
5. Other bugs:
•“Radius PrecisionColor 8”. Always operates in "gray" mode, even though
the driver pretends to support both color and gray. Try TimeVideo.
•Relax 19" Model 200 (.Color_Video_Display version 9288) doesn't support the optional getGamma Status call. The required getPageCnt (aka getPages) Status call erroneously returns a page count of 0 for all modes. Try TimeVideo or TestGDVideo. (Reported to Relax Tech. 1/19/93)
6. Video drivers that have been tested (version 0, unless noted otherwise):
Apple Mac Plus and SE 1-bit QuickDraw
Apple “Macintosh SE/30 Internal Video” (.Display_Video_Apple_MacSE/30 Video)
Apple “Toby frame buffer card” (.Display_Video_Apple_TFB version 5)
Apple “Mac II High-Resolution Video Card” (.Display_Video_Apple_HRVC)
Apple 8•24 “Macintosh Display Card” (.Display_Video_Apple_MDC)
